Optimizely Feature Experimentation

I would use Optimizely Feature Experimentation when we would like to run basic experiments where metrics to be tracked are impressions, revenue or clicks.

However, most of our experiments are tracking more complex metrics and this functionality is not enough. We still need to do work to analyse the data in our end.

It's very useful for turning features on or off. Introducing new features is simple. Opening up a feature for visitors matching specific conditions is also easy enough. It's not useful for offering different variations of a specific feature (AB-testing).

Optimizely Feature Experimentation is perfect for testing small copy changes, colour changes or the position of CTAs on a page. Optimizely Feature Experimentation is great to test small experiments to see what moves the needle but I would refrain from using it on larger, more complex experiments as it becomes too cumbersome to set up correctly.

To my experience, Optimizely should be used across all core software releases in platforms/apps and products that are customer facing regardless whether b2c or b2b. Even in cases of internal platforms with thousands of users, testing features or pre-releasing and monitoring functionality is key. Good to note that it is less appropriate in cases where minor releases, backend software improvements are referred to.

Well suited to teams who are shipping fast but wish to control roll outs to certain audiences, or to test the impact of releases for safety.

It's a great to test the customer behavior on new user interfaces and beta features. For example, before we invest a lot of money to develop a new capability or feature of the application, we want to create a quick proof-of-concept application. After the experiment, the business can a go or no-go decision.

As a digital marketing manager, I've found Optimizely extremely effective for refining web interfaces, testing marketing messages, and optimizing user journeys on large e-commerce platforms. Its strengths shine in environments with high user traffic. However, for smaller websites or those with limited traffic, it's less appropriate, as substantial user data is essential to achieve statistically reliable results and insights. This tool is best leveraged in scenarios where user engagement can be measured at scale.

Optimizely Feature Experimentation has all the features required to create feature flags and use them to control the release of software and to run hold backs. It's technically a good fit for these development and rollout use cases and helps avoid complicated version control alternative solutions. However, Optimizely Feature Experimentation is not easy or intuitive to use in my opinion. This is annoying from a usability and learning curve perspective. Worse, the unintuitive webapp can introduce risk where users may think they've toggled a feature flag for a target audience but instead have applied a different configuration (or don't have the flag active at all).

Optimizely is a useful tool for businesses looking to improve specific aspects of their website, such as the checkout process, product page layout, or homepage design. Although it may not be practical for small websites or businesses with limited online operations, Optimizely's Feature Experimentation is crucial for smooth feature rollouts and successful A/B testing. It significantly improves the online customer experience and optimizes conversion funnels. However, companies that do not heavily rely on ongoing feature deployment may not require all of Optimizely's functions.

When testing if a different color button, or different size, leads to a higher conversion rate.